Noise determination apparatus and method, and noise filter

ABSTRACT

A technique for determining noise is provided that suppresses misrecognition of significant components in an image as noise in any image captured under any condition. A noise determination apparatus for determining noise in image data that is input in units of frames decomposes the image data into frequency components, samples a predetermined number of data pieces for low-frequency components that have relatively low frequencies and a predetermined number of data pieces for high-frequency components that have relatively high frequencies from the frequency components, and analyzes whether or not the image data includes an edge image, on the basis of a ratio of high-frequency data to low-frequency data.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technique for determining noise, andin particular to a technique for determining noise in an image signal.

2. Description of the Background Art

Various techniques have conventionally been proposed to detect andremove noise in an image signal produced by an image sensor such as asolid-state image sensor.

For example, Japanese Patent Gazette No. 5105215 discloses a techniquefor detecting and removing noise included in chroma components.

Imaging devices using image sensors such as solid-state image sensorsmay be used as surveillance cameras, which are not limited for use inbright light and may be used in low light.

In the case of use in low light, it is difficult to discriminate betweennoise and significant components in the image. With conventionalnoise-detection techniques that unconditionally remove detected noise,which include Japanese Patent Gazette No. 5105215, there has been thepossibility that significant components in the image will bemisrecognized as noise and removed.

In addition, there is another problem in that although the ratio ofnoise is high in low light, the conventional noise-detection techniques,which include Japanese Patent Gazette No. 5105215, cannot effectivelydetermine noise and accordingly cannot remove noise satisfactorily.

SUMMARY OF THE INVENTION

The present invention has been achieved in order to solve problems asdescribed above, and it is an object of the present invention to providea technique for determining noise, which is capable of determining noisewhile suppressing misrecognition of significant components as noise inany image captured under any condition.

A first aspect of a noise determination apparatus according to thepresent invention is a noise determination apparatus for determiningnoise in image data that is input in units of frames. The noisedetermination apparatus includes circuitry to decompose the image datainto frequency components, sample a predetermined number of data piecesfor low-frequency components that have relatively low frequencies and apredetermined number of data pieces for high-frequency components thathave relatively high frequencies from the frequency components, andanalyze whether or not the image data includes an edge image, on thebasis of a ratio of high-frequency data to low-frequency data.

In a second aspect of the noise determination apparatus according to thepresent invention, the circuitry includes a frequency decomposer thatdivides one frame's worth of the image data into matrices of apredetermined number of pixels, the matrices serving as data blocks, anddecomposes each data block into frequency components, a frequencyanalyzer that samples a predetermined number of low-frequency componentsand a predetermined number of high-frequency components from thefrequency components of the data block obtained by the frequencydecomposer to use the low-frequency components as the low-frequency dataand use the high-frequency components as the high-frequency data, andanalyzes whether or not the data block includes an edge image, on thebasis of a ratio of the high-frequency data to the low-frequency data,and a filter characteristic selector that selects a filtercharacteristic of a filter that removes noise, on the basis of a resultof the analysis performed by the frequency analyzer.

In a third aspect of the noise determination apparatus according to thepresent invention, the frequency analyzer sets a first sampling area inwhich the low-frequency data includes a greater number of horizontalfrequency components than vertical frequency components, in a case wherea ratio of the high-frequency data to the low-frequency data in thefirst sampling area is less than or equal to a predetermined thresholdvalue, the frequency analyzer analyzes that the data block includes ahorizontal edge image, and the filter characteristic selector selects afilter characteristic that does not remove the horizontal edge image.

In a forth aspect of the noise determination apparatus according to thepresent invention, the frequency analyzer sets a second sampling area inwhich the low-frequency data includes a greater number of verticalfrequency components than horizontal frequency components, in a casewhere a ratio of the high-frequency data to the low-frequency data inthe second sampling area is less than or equal to a predeterminedthreshold value, the frequency analyzer analyzes that the data blockincludes a vertical edge image, and the filter characteristic selectorselects a filter characteristic that does not remove the vertical edgeimage.

In a fifth aspect of the noise determination apparatus according to thepresent invention, the frequency analyzer sets a third sampling area inwhich the low-frequency data includes equal numbers of verticalfrequency components and horizontal frequency components, in a casewhere a ratio of the high-frequency data to the low-frequency data inthe third sampling area is less than or equal to a predeterminedthreshold value, the frequency analyzer analyzes that the data blockincludes an edge image, and the filter characteristic selector selects afilter characteristic that does not remove the edge image.

In a sixth aspect of the noise determination apparatus according to thepresent invention, the frequency decomposer decomposes a data block intofrequency components through a discrete cosine transform, and thefrequency analyzer normalizes alternating current components amongtransform coefficients obtained through the discrete cosine transformperformed by the frequency decomposer by dividing the alternatingcurrent components by a direct current component among the transformcoefficients, and uses the normalized alternating current components asthe low-frequency data and the high-frequency data.

In a seventh aspect of the noise determination apparatus according tothe present invention, the circuitry further includes an isolatedfrequency analyzer that analyzes whether or not a frequency component isisolated from other frequency components to determine presence orabsence of an isolated frequency. The isolated frequency analyzeranalyzes whether or not the data block includes an edge image that isunable to be analyzed by the frequency analyzer, on the basis of thepresence or absence of the isolated frequency, and in a case where thedata block includes an edge image that is unable to be analyzed by thefrequency analyzer, the filter characteristic selector selects a filtercharacteristic that does not remove the edge image.

In an eighth aspect of the noise determination apparatus according tothe present invention, the frequency decomposer decomposes a data blockinto frequency components through a discrete cosine transform, and theisolated frequency analyzer normalizes alternating current componentsamong transform coefficients that are obtained through the discretecosine transform performed by the frequency decomposer by dividing thealternating current components by a direct current component among thetransform coefficients, calculates an average value of all normalizedalternating current components, calculates, for each of frequencycomponents of the normalized alternating current components, the numberof surrounding frequency components that have values greater than theaverage value, calculates the number of frequency components for whichthe calculated number of surrounding frequency components is less than apredetermined first threshold value, determines whether or not thecalculated number of frequency components is less than a predeterminedsecond threshold value, and if the calculated number of frequencycomponents is less than the second threshold value, analyzes that thedata block includes an edge image that is unable to be analyzed by thefrequency analyzer.

In a ninth aspect of the noise determination apparatus according to thepresent invention, Bayer data serves as the image data.

A tenth aspect of the noise determination apparatus according to thepresent invention is a noise determination apparatus for determiningpresence or absence of noise in image data that is input in units offrames. The noise determination apparatus includes circuitry configuredto decompose the image data into frequency components, sample apredetermined number of data pieces for low-frequency components thathave relatively low frequencies and a predetermined number of datapieces for high-frequency components that have relatively highfrequencies from the frequency components, and determine the presence orabsence of noise on the basis of a ratio of high-frequency data tolow-frequency data.

In an eleventh aspect of the noise determination apparatus according tothe present invention, the circuitry includes a frequency decomposerthat divides one frame's worth of the image data into matrices of apredetermined number of pixels, the matrices serving as data blocks, anddecomposes each data block into frequency components, and a frequencyanalyzer that samples a predetermined number of low-frequency componentsand a predetermined number of high-frequency components from thefrequency components of the data block obtained by the frequencydecomposer to use the low-frequency components as the low-frequency dataand use the high-frequency components as the high-frequency data, andanalyzes the presence or absence of noise on the basis of whether or nota ratio of the high-frequency data to the low-frequency data is greaterthan a predetermined first threshold value.

In an twelfth aspect of the noise determination apparatus according tothe present invention, the circuitry includes a single-frame noisedetermination unit that determines that the one frame's worth of theimage data includes noise, in a case where the number of the data blocksthat are determined to include noise by the frequency analyzer isgreater than a predetermined threshold value.

In a thirteenth aspect of the noise determination apparatus according tothe present invention, the circuitry includes a multiple-frame noisedetermination unit that determines that a moving image includes noise,in a case where a plurality of frames that are determined to includenoise by the single-frame noise determination unit satisfy apredetermined determination condition.

In a fourteenth aspect of the noise determination apparatus according tothe present invention, the circuitry includes a frequency decomposerthat divides one frame's worth of the image data into matrices of apredetermined number of pixels, the matrices serving as data blocks, anddecomposes each data block into frequency components, and a frequencyanalyzer that samples a predetermined number of low-frequency componentsand a predetermined number of high-frequency components from thefrequency components of the data block obtained by the frequencydecomposer to use the low-frequency components as the low-frequency dataand use the high-frequency components as the high-frequency data,generates a data-integrated matrix by adding data pieces ofcorresponding frequency components in all of the data blocks thatconstitute the one frame's worth of the image data, and analyzes thepresence or absence of noise on the basis of whether or not a ratio ofthe high-frequency data to the low-frequency data in the data-integratedmatrix is greater than a predetermined first threshold value.

In a fifteenth aspect of the noise determination apparatus according tothe present invention, the circuitry includes a multiple-frame noisedetermination unit that determines that a moving image includes noise,in a case where a plurality of frames that are determined to includenoise by the frequency analyzer satisfies a predetermined determinationcondition.

In a sixteenth aspect of the noise determination apparatus according tothe present invention, the determination condition is defined by a ratioof frames that include noise to a plurality of frames that constitute amoving image.

In a seventeenth aspect of the noise determination apparatus accordingto the present invention, the determination condition is defined by acontinuous number of frames that include noise among a plurality offrames that constitute a moving image.

In an eighteenth aspect of the noise determination apparatus accordingto the present invention, the frequency decomposer decomposes a datablock into frequency components through a discrete cosine transform, andthe frequency analyzer normalizes alternating current components amongtransform coefficients obtained through the discrete cosine transformperformed by the frequency decomposer by dividing the alternatingcurrent components by a direct current component among the transformcoefficients, and uses the normalized alternating current components asthe low-frequency data and the high-frequency data.

In a nineteenth aspect of the noise determination apparatus according tothe present invention, the frequency analyzer analyzes the presence orabsence of noise by further determining whether or not thehigh-frequency data that has been normalized is greater than apredetermined second threshold value.

In a twentieth aspect of the noise determination apparatus according tothe present invention, Bayer data serves as the image data.

A first aspect of a noise determination method according to the presentinvention is performed by a noise determination apparatus fordetermining noise in image data that is input in units of frames. Themethod includes the steps of (a) decomposing the image data intofrequency components, and (b) sampling a predetermined number of datapieces for low-frequency components that have relatively low frequenciesand a predetermined number of data pieces for high-frequency componentsthat have relatively high frequencies from the frequency componentsobtained in the step (a), and analyzing whether or not the image dataincludes an edge image, on the basis of a ratio of high-frequency datato low-frequency data.

A second aspect of the noise determination method according to thepresent invention is a noise determination method for determiningpresence or absence of noise in image data that is input in units offrames. The method includes the steps of (a) decomposing the image datainto frequency components, and (b) sampling a predetermined number ofdata pieces for low-frequency components that have relatively lowfrequencies and a predetermined number of data pieces for high-frequencycomponents that have relatively high frequencies from the frequencycomponents obtained in the step (a), and determining the presence orabsence of noise on the basis of a ratio of high-frequency data tolow-frequency data.

A noise filter according to an aspect of the present invention filtersfrequency components that are obtained by decomposing image data.

The noise determination apparatus according to the present invention iscapable of preventing edge components from being removed throughfiltering and is capable of suppressing removal of significantcomponents in an image along with noise. The apparatus is also capableof relatively simply and easily analyzing the presence or absence ofnoise in any image captured under any condition.

These and other objects, features, aspects and advantages of the presentinvention will become more apparent from the following detaileddescription of the present invention when taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of anoise-removing apparatus that includes a noise determination apparatusaccording to a preferred embodiment of the present invention;

FIG. 2 is a block diagram illustrating a configuration of the noisedetermination apparatus according to the preferred embodiment of thepresent invention;

FIG. 3 is a block diagram illustrating a configuration of a filtercircuit that removes noise on the basis of a result of determinationperformed by the noise determination apparatus according to thepreferred embodiment of the present invention;

FIGS. 4 to 7 are flowcharts for explaining processing for determiningnoise, performed by the noise determination apparatus according to thepreferred embodiment of the present invention;

FIGS. 8 to 10 illustrate matrices of DCT coefficients, each matrixcorresponding to a Bayer data block of 8×8 pixels;

FIGS. 11 to 18 illustrate structures of quantization matrix tables usedto remove noise;

FIG. 19 is a block diagram illustrating a configuration of anoise-removing apparatus that includes a noise determination apparatusaccording to another preferred embodiment of the present invention;

FIG. 20 is a block diagram illustrating a configuration of the noisedetermination apparatus according to the preferred embodiment of thepresent invention;

FIG. 21 is a flowchart for explaining processing for determining noise,performed by the noise determination apparatus according to thepreferred embodiment of the present invention;

FIG. 22 illustrates a matrix of DCT coefficients, the matrixcorresponding to a Bayer data block of 8×8 pixels;

FIG. 23 is a block diagram illustrating a variation of the noisedetermination apparatus according to the preferred embodiment of thepresent invention;

FIG. 24 is a flowchart for explaining processing for determining noise,performed by the variation of the noise determination apparatusaccording to the preferred embodiment of the present invention; and

FIGS. 25 and 26 show examples of how to set sampling areas.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

First Preferred Embodiment

FIG. 1 is a block diagram illustrating a configuration of anoise-removing apparatus 100 that includes a noise determinationapparatus according to a first preferred embodiment of the presentinvention.

As illustrated in FIG. 1, the noise-removing apparatus 100 includes anoise determination apparatus 1 that receives input of image data PD anddetermines whether or not the image data PD includes noise, and a filtercircuit 2 that removes noise in the image data PD.

The filter circuit 2 receives input of a characteristic selection signalCS and the image data PD from the noise determination apparatus 1 andoutputs the image data PD, either directly or after filtering.

FIG. 2 is a block diagram illustrating a configuration of the noisedetermination apparatus 1. As illustrated in FIG. 2, the noisedetermination apparatus 1 receives input of Bayer data serving as theimage data PD and temporarily stores the Bayer data in a line memorygroup 11. As will be described later, the noise determination apparatus1 processes one frame's worth of the image data PD in units of matricesof 8×8 or 4×4 pixels. The line memory group 11 thus includes seven linememories when the data is processed in units of 8×8 pixels, and itincludes three line memories when the image data is processed in unitsof 4×4 pixels.

The Bayer data stored in the line memory group 11 is transmitted to adiscrete cosine transform (DCT) unit 12 in units of 4×4 or 8×8 pixels infirst-in, first-out (FIFO) order. The data is transmitted in successionuntil four or eight lines of the Bayer data have been transmitted.

The DCT unit 12 (frequency decomposer) performs a discrete cosinetransform on the received Bayer data (hereinafter, referred to as a“Bayer data block”) to decompose the data into frequency components, andoutputs transform coefficients of the obtained frequency components.

The transform coefficients output from the DCT unit 12 are analyzed by afrequency analyzer 13 that analyzes noise included in the single framein units of Bayer data blocks. The result of the analysis is transmittedto a filter characteristic selector 15.

The output of the DCT unit 12 and the result of the analysis performedby the frequency analyzer 13 are also transmitted to an isolatedfrequency analyzer 14 that analyzes an isolated frequency and transmitsa result of the analysis to the filter characteristic selector 15.

The filter characteristic selector 15 generates a quantization matrixtable for determining a filter characteristic, on the basis of theresults of the analyses performed by the frequency analyzer 13 and theisolated frequency analyzer 14, and transmits, to the filter circuit 2,a filter characteristic selection signal FS that indicates aquantization matrix table and a control signal CS for selecting whetherto filter the image data PD before output or to output the image data PDdirectly.

Note that the noise determination apparatus 1 is configured of, forexample, a central processing unit (CPU), a random-access memory (RAM),a read-only memory (ROM), and a wired logic circuit. The functionalityof the noise determination apparatus 1 is implemented by the CPUexecuting a program that is stored in and read out of the ROM.

FIG. 3 illustrates a configuration of the filter circuit 2. Asillustrated in FIG. 3, the filter circuit 2 includes a selection circuit20 that selects whether to filter the image data PD before output or tooutput the image data PD directly, a DCT unit 21 that performs a DCT onthe image data PD, a quantization unit 22 that performs quantizationprocessing on the data obtained from the DCT, an inverse quantizationunit 23 that performs inverse quantization processing on the quantizeddata, and an inverse DCT unit 24 that performs an inverse DCT on thedata obtained from the inverse quantization processing.

A selection operation performed by the selection circuit 20 iscontrolled by the control signal CS transmitted from the filtercharacteristic selector 15, and in the case of performing filtering, theimage data PD is transmitted to the DCT unit 21.

Noise Determination Processing

Next, processing for determining edges among the processing fordetermining noise performed by the noise determination apparatus 1 willbe described with reference to the flowcharts in FIGS. 4 to 7.

Determination of Horizontal Edges

First, determining horizontal edges will be described with reference toFIG. 4. Note that the following describes processing from the frequencyanalyzer 13 onward in FIG. 2, and a description of DCT processing, whichis a known technique, will be omitted.

In step S1, the frequency analyzer 13 normalizes alternating current(AC) components in a matrix (corresponding to a Bayer data block of 8×8pixels) of transform coefficients (DCT coefficients) output from the DCTunit 12, by dividing the AC components by a direct current (DC)component in the matrix.

Here, FIG. 8 illustrates the matrix of DCT coefficients, whichcorresponds to a Bayer data block of 8×8 pixels. In FIG. 8, the firstelement (at the top left corner) of the matrix is the DC component,elements along the horizontal direction of the matrix are verticalfrequency components (V), and elements along the vertical direction ofthe matrix are horizontal frequency components (H). AC components thatare closer to the DC component have lower frequencies, and AC componentsthat are closer to a corner that is diagonally opposite to the DCcomponent in the matrix have higher frequencies.

In FIG. 8, a sampling area of low-frequency components that are ACcomponents located close to the DC component is shown as a sampling areaLSR (low-frequency sampling area), and a sampling area of high-frequencycomponents that are AC components located close to the corner diagonallyopposite to the DC component is shown as a sampling area HSR(high-frequency sampling area).

Note that the sampling areas can be set arbitrarily, and the frequencyanalyzer 13 sets a sampling area HLSR (horizontal low-frequency samplingarea) that includes a greater number of horizontal frequency components(H) and a smaller number of vertical frequency components (V) asillustrated in FIG. 9. The frequency analyzer 13 then calculates a ratio(Σ high frequencies/Σ horizontal low frequencies) of the total number ofDCT coefficients normalized in the sampling area HSR (Σ highfrequencies) to the total number of DCT coefficients normalized in thesampling area HLSR (Σ horizontal low frequencies), and compares thecalculated ratio with a predetermined threshold value A (step S2).

If the ratio exceeds the threshold value A, the procedure proceeds tostep S4. If the ratio is less than or equal to the threshold value A, itis determined that the Bayer data block to be processed corresponds toan image with a great number of horizontal edges, and the procedureproceeds to step S3. Note that the threshold value A can be setarbitrarily.

Here, horizontal edges refer to edges along the vertical direction ofthe image, and a great number of horizontal edges can possibly cause theimage to be misrecognized to include noise. The determination ofhorizontal edges is performed in order for horizontal edges not to bemisrecognized as noise and removed.

Upon receiving the analysis result indicating that the Bayer data blockto be processed corresponds to an image with a great number ofhorizontal edges, the filter characteristic selector 15 selects such afilter characteristic of the filter circuit 2 that the filter circuit 2serves as a filter that is weak enough to conform to the image with agreat number of horizontal edges (step S3).

In other words, the filter characteristic selector 15 sets aquantization matrix table with which horizontal edges will not beremoved as noise, and transmits that quantization matrix table to thefilter circuit 2.

Note that, in the case of selecting a filter characteristic in step S3,the selection circuit 20 of the filter circuit 2 is controlled so thatfiltering is performed on the image data PD that corresponds to theBayer data block to be processed.

On the other hand, if it is determined in step S2 that the Bayer datablock to be processed does not correspond to an image with a greatnumber of horizontal edges, filtering is not performed on the image dataPD.

Then, it is confirmed in step S4 whether or not the determination ofnoise in step S2 has been performed on all Bayer data blocks in thesingle frame, and if the noise determination processing has beenperformed on all of the Bayer data blocks, the determination processingand filtering on that frame ends. If there is another Bayer data blockthat has not yet undergone the determination processing, the processingfrom step S1 onward is repeated.

Determination of Vertical Edges

Next is a description of determining vertical edges with reference toFIG. 5. Note that step S11 is the same processing step as step S1described with reference to FIG. 4, and therefore a description thereofwill be omitted.

The frequency analyzer 13 sets a sampling area VLSR (verticallow-frequency sampling area) that includes a greater number of verticalfrequency components (V) and a smaller number of horizontal frequencycomponents (H) as illustrated in FIG. 10. The frequency analyzer 13 thencalculates a ratio (Σ high frequencies/Σ vertical low frequencies) ofthe total number of DCT coefficients normalized in the sampling area HSR(Σ high frequencies) to the total number of DCT coefficients normalizedin the sampling area VLSR (Σ vertical low frequencies), and compares thecalculated ratio with a predetermined threshold value B (step S12).

If the ratio exceeds the threshold value B, the procedure proceeds tostep S14. If the ratio is less than or equal to the threshold value B,it is determined that the Bayer data block to be processed correspondsto an image with a great number of vertical edges, and the procedureproceeds to step S13. Note that the threshold value B can be setarbitrarily.

Here, vertical edges refer to edges along the horizontal direction ofthe image, and a great number of vertical edges can possibly cause theimage to be misrecognized to include noise. The determination ofvertical edges is performed in order for vertical edges not to bemisrecognized as noise and removed.

Upon receiving the analysis result indicating that the Bayer data blockto be processed corresponds to an image with a great number of verticaledges, the filter characteristic selector 15 selects such a filtercharacteristic of the filter circuit 2 that the filter circuit 2 servesas a filter that is weak enough to conform to the image with a greatnumber of vertical edges (step S13).

In other words, the filter characteristic selector 15 sets aquantization matrix table with which vertical edges will not be removedas noise, and transmits that quantization matrix table to the filtercircuit 2.

Note that, in the case of selecting a filter characteristic in step S13,the selection circuit 20 of the filter circuit 2 is controlled so thatfiltering is performed on the image data PD that corresponds to theBayer data block to be processed.

On the other hand, if it is determined in step S12 that the Bayer datablock to be processed does not correspond to an image with a greatnumber of vertical edges, filtering is not performed on the image dataPD.

Then, it is confirmed in step S14 whether or not the noise determinationin step S12 has been performed on all Bayer data blocks in the singleframe. If the noise determination processing has been performed on allof the Bayer data blocks, the determination processing and filtering onthat frame ends. If there is another Bayer data block that has not yetundergone the determination processing, the processing from step S11onward is repeated.

Determination of Horizontal and Vertical Edges

Next is a description of determining edges with reference to FIG. 6.Note that step S21 is the same processing step as step S1 described withreference to FIG. 4, and therefore a description of thereof will beomitted.

The frequency analyzer 13 sets the sampling areas LSR and HSR that eachinclude equal numbers of vertical frequency components (V) andhorizontal frequency components (H) as illustrated in FIG. 8. Thefrequency analyzer 13 then calculates a ratio (Σ high frequencies/Σ lowfrequencies) of the total number of DCT coefficients normalized in thesampling area HSR (Σ high frequencies) to the total number of DCTcoefficients normalized in the sampling area LSR (Σ low frequencies),and compares the calculated ratio with a predetermined threshold value C(step S22).

If the ratio exceeds the threshold value C, the procedure proceeds tostep S24. If the ratio is less than or equal to the threshold value C,it is determined that the Bayer data block to be processed correspondsto an image with a great number of edges (which includes both verticaland horizontal edges), and the procedure proceeds to step S23. Note thatthe threshold value C can be set arbitrarily.

Here, a great number of edges can possibly cause the image to bemisrecognized to include noise. The determination of horizontal andvertical edges is performed in order for edges not to be misrecognizedas noise and removed.

Upon receiving the analysis result indicating that the Bayer data blockto be processed corresponds to an image with a great number of edges,the filter characteristic selector 15 selects such a filtercharacteristic of the filter circuit 2 that the filter circuit 2 servesas a filter that is weak enough to conform to the image with a greatnumber of edges (step S23).

In other words, the filter characteristic selector 15 sets aquantization matrix table with which edges will not be removed as noise,and transmits that quantization matrix table to the filter circuit 2.

Note that, in the case of selecting a filter characteristic in step S23,the selection circuit 20 of the filter circuit 2 is controlled so thatfiltering is performed on the image data PD that corresponds to theBayer data block to be processed.

On the other hand, if it is determined in step S22 that the Bayer datablock to be processed does not correspond to an image with a greatnumber of edges, filtering is not performed on the image data PD.

Then, it is confirmed in step S24 whether or not the noise determinationin step S22 has been performed on all Bayer data blocks in the singleframe. If the noise determination processing has already been performedon all of the Bayer data blocks, the determination processing andfiltering on that frame ends. If there is another Bayer data block thathas not yet undergone the determination processing, the processing fromstep S21 onward is repeated.

Combination of Edge Determination

The different types of edge determination described with reference toFIGS. 4 to 6 may each be used independently, or some or all of them maybe combined.

As one example, the determination of horizontal edges is performed instep S2 of FIG. 4, and if it is determined that the Bayer data block tobe processed does not correspond to an image with a great number ofhorizontal edges, then the determination of vertical edges is performedas in step S12 of FIG. 5. If it is determined that the Bayer data blockto be processed does not correspond to an image with a great number ofvertical edges, then the determination of edges is performed as in stepS22 of FIG. 6.

Combining different types of edge determination in this way allows animage with various types of edges to be filtered on the basis of anappropriate filter characteristic.

Analysis of Isolated Frequency

Next is a description of analysis of isolated frequencies, performed bythe isolated frequency analyzer 14, with reference to the flowchart inFIG. 7.

The isolated frequencies refers to frequency components that areisolated from other frequency components. Removing isolated frequenciesleads to noise reduction, but the presence of edges of special forms canpossibly be misrecognized as the presence of a great number of isolatedfrequencies.

The isolated frequency analyzer 14 confirms the presence or absence ofisolated frequencies, and if there are a great number of isolatedfrequencies, the isolated frequency analyzer 14 transmits thatinformation to the filter characteristic selector 15.

Note that the isolated frequency analyzer 14 also receives the result ofthe analysis performed by the frequency analyzer 13, and analyzesisolated frequencies only when the analysis result from the frequencyanalyzer 13 indicates that any type of edges has not been detected.

In step S31 of FIG. 7, the isolated frequency analyzer 14 normalizes ACcomponents in the matrix of DCT coefficients output from the DCT unit12, by dividing the AC components by the DC component in the matrix.

The isolated frequency analyzer 14 then calculates an average value ofall normalized AC components (in the present example, 63 AC components)(step S32).

Next, in step S33, the isolated frequency analyzer 14 specifiesfrequency components that each have eight frequency components in theperiphery thereof, from among the 63 AC components. This is the processfor confirming whether or not each of the 63 frequency components or ACcomponents has eight frequency components in the periphery thereof, inother words, the process for specifying a frequency component that haseight frequency components in the periphery thereof, i.e., a centralfrequency component that is surrounded by eight frequency components.Note that frequency components to be specified are not the ones thatconstitute the four sides of the Bayer data block, so the frequencycomponents that constitute the four sides of the Bayer data block may beexcluded from those to be processed in step S33.

If frequency components (central frequency components) that are eachsurrounded by eight frequency components are specified in step S33, theisolated frequency analyzer 14 calculates, for each of the centralfrequency components, the number of surrounding frequency componentsthat have values greater than the average value calculated in step S32from among the eight surrounding frequency components, and thencalculates the number of central frequency components for which thecalculated number of surrounding frequency components is less than apredetermined threshold value D. Such central frequency components canbe said to be isolated frequencies, but there are two possiblesituations where the isolated frequencies merely represent noise andwhere the isolated frequencies represent an edge of special form. Thus,it is determined whether or not the calculated number of centralfrequency components is less than a predetermined threshold value F(step S34), and if the number is greater than or equal to the thresholdvalue F, it is determined that the Bayer data block to be processedcorresponds to an image with an edge of special form, instead of animage that includes noise.

In other words, a small number of AC components that have values greaterthan the average value suggests the presence of isolated frequencies,and a great number of isolated frequencies suggests the possibility ofthe presence of an edge of special form that cannot be analyzed by thefrequency analyzer 13. Thus, if a plurality of central frequencycomponents specified in step S33 includes a total of F or more centralfrequency components that each have less than D surrounding frequencycomponents that have values greater than the average value (No in stepS34), it is determined that the Bayer data block to be processedcorresponds to an image with an edge of special form, and the procedureproceeds to step S35. Note that the threshold values D and F can be setarbitrarily.

On the other hand, if it is determined in step S34 that the calculatednumber of central frequency components is less than F (Yes), theprocedure proceeds to step S36.

In step S36, frequency components that each have five frequencycomponents in the periphery thereof are specified from among the 63 ACcomponents. This is the process for confirming whether or not each ofthe 63 AC components has five frequency components in the peripherythereof, in other words, the process for specifying frequency componentsthat each have five frequency components in the periphery thereof, i.e.,frequency components that are not the ones located at the corners, fromamong the frequency components that constitute the four sides of theBayer data block. Note that frequency components to be specified areonly the ones that constitute the four sides of the Bayer data block,and therefore the other frequency components may be excluded from thoseto be processed in step S36.

If frequency components (edge frequency components) that are eachsurround by five frequency components are specified in step S36, theisolated frequency analyzer 14 calculates, for each of the edgefrequency components, the number of surrounding frequency componentsthat have values greater than the average value calculated in step S32from among the five surrounding frequency components, and thencalculates the number of edge frequency components for which thecalculated number of surrounding frequency components is less than apredetermined threshold value E. Such edge frequency components can besaid to be isolated frequencies, but there are two possible situationswhere the isolated frequencies merely represent noise and where theisolated frequencies represent an edge of special form. Thus, it isdetermined whether or not the calculated number of edge frequencycomponents is less than a predetermined threshold value G (step S37),and if the number is greater than or equal to the threshold value G, itis determined that the Bayer data block to be processed corresponds toan image with an edge of special form, instead of an image that includesnoise.

In other words, a small number of AC components that have values greaterthan the average values suggests the presence of isolated frequencies,and a great number of isolated frequencies suggests the possibility ofthe presence of an edge of special form that cannot be analyzed by thefrequency analyzer 13. Thus, if a plurality of edge frequency componentsspecified in step S36 includes a total of G or more edge frequencycomponents that each have less than E surrounding frequency componentsthat have values greater than the average value (No in step S37), it isdetermined that the Bayer data block to be processed corresponds to animage with an edge of special form, and the procedure proceeds to stepS35. Note that the threshold values E and G can be set arbitrarily.

On the other hand, if it is determined in step S37 that the calculatednumber of edge frequency components is less than G (YES), it isdetermined that the Bayer data block to be processed does not correspondto an image with an edge of special form, and the procedure proceeds tostep S38.

In step S35, upon receiving, from the isolated frequency analyzer 14,the analysis result indicating that the Bayer data block to be processedcorresponds to an image with an edge of special form, the filtercharacteristic selector 15 selects such a filter characteristic of thefilter circuit 2 that the filter circuit 2 serves as a filter that isweak enough to leave the edge.

In other words, the filter characteristic selector 15 sets aquantization matrix table with which an edge of special form will not beremoved, and transmits that quantization matrix table to the filtercircuit 2.

Note that, in the case of selecting a filter characteristic in step S35,the selection circuit 20 of the filter circuit 2 is controlled so thatfiltering is performed on the image data PD that corresponds to theBayer data block to be processed.

Then, it is confirmed in step S39 whether or not the analysis ofisolated frequencies have been performed on all of the Bayer datablocks, and if the analysis have been completed, a series of processesends. If there is another Bayer data block that has not been analyzedyet, the processing from step S31 onward is repeated.

On the other hand, when having received the analysis result indicatingthat the Bayer data block to be processed does not correspond to animage with an edge of special form from the isolated frequency analyzer14, the filter characteristic selector 15 selects such a filtercharacteristic of the filter circuit 2 that the filter circuit 2 servesas a filter that conforms to an image with no edges (step S38).

Then, it is confirmed in step S39 whether or not the analysis ofisolated frequencies have been performed on all of the Bayer datablocks, and if the analysis has been completed, a series of processesends. If there is another Bayer data block that has not been analyzedyet, the processing from step S31 onward is repeated.

Here, the accuracy of determination can be improved by performing theabove-described determination processing on each of four colors of theinput Bayer data, namely, green of blue side (Gb), green of red side(Gr), red (R), and blue (B). Alternatively, the above-describeddetermination processing may be performed on only Gb or Gr data becausethe green components include luminance information.

Combination with Typical Edge Detection Technique

The Sobel method, for example, is typically used to detect edges in acertain area of an image.

By combining the edge detection techniques described with reference toFIGS. 4 to 7 and the edge detection technique using the Sobel method, animage that includes more various types of edges can be filtered based onan appropriate filter characteristic.

For example, if any type of edges is not detected by any of the edgedetection techniques described with reference to FIGS. 4 to 7 (or acombination of the edge detection techniques), the Sobel method is usedto detect edges, and if there are few edges that have been detected, thefilter characteristic of the filter circuit 2 is selected so that thefilter circuit 2 serves as a strong filter in consideration of thedetermination result that the image does not include edges.

In other words, since there are few edges, a quantization matrix tablewith which noise will be removed is set and transmitted to the filtercircuit 2 without consideration of edges. This quantization matrix tablerealizes the functionality of a filter that removes both high and lowfrequencies.

Note that if a great number of edges are detected through the edgedetection using the Sobel method, the filter characteristic of thefilter circuit 2 is selected so that the filter circuit 2 serves as astrong filter in consideration of the determination result that theimage includes some types of edges.

Filtering

Next is a description of filtering performed by the filter circuit 2. Asillustrated in FIG. 3, the filter circuit 2 includes the DCT unit 21that performs a DCT on the image data PD, the quantization unit 22 thatperforms quantization processing on the data obtained from the DCT, theinverse quantization unit 23 that performs inverse quantizationprocessing on the quantized data, and the inverse DCT unit 24 thatperforms an inverse DCT on the data obtained from the inversequantization processing.

The quantization unit 22 quantizes data obtained from the DCT bydividing the data using a predetermined quantization matrix table. Thisprocess corresponds to filtering. Filter strength can be changedarbitrarily by changing the quantization matrix table. A signal used todetermine a quantization matrix table is the filter characteristicselection signal FS. The filter characteristic selection signal FS maybe used to select a quantization matrix table from among a plurality ofpre-provided quantization matrix tables, or it may indicate aquantization matrix table that is generated by the filter characteristicselector 15.

Note that the image data can be restored by the inverse quantizationunit 23 performing inverse quantization processing on the quantized dataand the inverse DCT unit 24 performing an inverse DCT on the dataobtained from the inverse quantization processing. This restored imagedata corresponds to filtered image data.

The inverse quantization processing performed by the inversequantization unit 23, which is the process for multiplying the quantizeddata using a predetermined quantization matrix table, also uses aquantization matrix table. A signal used to determine this quantizationmatrix table is the filter characteristic selection signal FS. Thefilter characteristic selection signal FS may be used to select aquantization matrix table from among a plurality of pre-providedquantization matrix tables, or it may indicate a quantization matrixtable that is generated by the filter characteristic selector 15. Notethat the tables used by the quantization unit 22 and the inversequantization unit 23 are not the same, but since they are used as apair, the same filter characteristic selection signal FS is provided tothe quantization unit 22 and the inverse quantization unit 23 in FIG. 3.

Examples of Quantization Matrix Table

The following describes examples of the quantization matrix table withreference to FIGS. 11 to 18. Note that the quantization matrix table maybe provided in advance and used by the filter characteristic selector15, or it may be generated each time by the filter characteristicselector 15 on the basis of the analysis results received from thefrequency analyzer 13 and the isolated frequency analyzer 14.

Table for Horizontal Edges

FIG. 11 illustrates a quantization matrix table that is provided to thefilter circuit 2 in order not to remove horizontal edges as noise whenthe determination of horizontal edges described with reference to FIG. 4shows that the Bayer data block to be processed corresponds to an imagewith a great number of horizontal edges.

As illustrated in FIG. 11, the quantization matrix table givessignificant numerical values to only components in a low-frequency areathat is set to include a greater number of horizontal frequencycomponents than vertical frequency components as in the matrix of DCTcoefficients in FIG. 9, and gives a maximum value of “255” to all theremaining components in the other area. Note that the significantnumerical values can be set arbitrarily.

This provides a filter characteristic that leaves only AC componentsthat are given significant numerical values and removes the other ACcomponents.

Table for Vertical Edges

FIG. 12 illustrates a quantization matrix table that is provided to thefilter circuit 2 in order not to remove vertical edges as noise when thedetermination of vertical edges described with reference to FIG. 5 showsthat the Bayer data block to be processed corresponds to an image with agreat number of vertical edges.

As illustrated in FIG. 12, the quantization matrix table givessignificant numerical values to only components in a low-frequency areathat is set to include a greater number of vertical frequency componentsthan horizontal frequency components as in the matrix of DCTcoefficients in FIG. 10, and gives a maximum value of “255” to all theremaining components in the other area.

This provides a filter characteristic that leaves only AC componentsthat are given significant numerical values and removes the other ACcomponents.

Table for Horizontal and Vertical Edges

FIG. 13 illustrates a quantization matrix table that is provided to thefilter circuit 2 in order not to remove horizontal and vertical edges asnoise when the edge determination described with reference to FIG. 6shows that the Bayer data block to be processed corresponds to an imagewith a great number of both horizontal and vertical edges.

As illustrated in FIG. 13, the quantization matrix table givessignificant numerical values to only components in a low-frequency areathat is set to include equal numbers of horizontal frequency componentsand vertical frequency components as in the matrix of DCT coefficientsin FIG. 8, and gives a maximum value of “255” to all the remainingcomponents in the other area.

This provides a filter characteristic that leaves only AC componentsthat are given significant numerical values and removes the other ACcomponents.

Table for Edges Obtained through Analysis of Isolated Frequencies

FIG. 14 illustrates a quantization matrix table that is provided to thefilter circuit 2 in order not to remove an edge of special form as noisewhen the analysis of isolated frequencies described with reference toFIG. 7 shows that the Bayer data block to be processed corresponds to animage that includes an edge of special form at an edge or in any otherportion.

As illustrated in FIG. 14, the quantization matrix table is set to havea low-frequency area that includes equal numbers of horizontal frequencycomponents and vertical frequency components, the area being smallerthan the corresponding area of the quantization matrix table in FIG. 13.This increases the number of AC components to be removed, thus providinga stronger filter characteristic.

Table for Edges Detected Using Sobel Method

FIG. 15 illustrates a quantization matrix table that is provided to thefilter circuit 2 to remove noise without consideration of edges when fewedges are detected even using the Sobel method.

As illustrated in FIG. 15, the quantization matrix table is set to havea low-frequency area that includes equal numbers of horizontal frequencycomponents and vertical frequency components, the area being smallerthan the corresponding area of the quantization matrix table in FIG. 14.This increases the number of AC components to be removed, thus providinga stronger filter characteristic.

Table for Edges Detected Using Combination of Edge DeterminationTechniques

As described previously, in the case of using a combination of differenttypes of edge determination techniques described with reference to FIGS.4 to 6, a quantization matrix table different from that used in the caseof using a single edge determination technique is used.

For example, FIG. 16 illustrates a quantization matrix table that isprovided to the filter circuit 2 when the three types of edgedetermination techniques described with reference to FIGS. 4 to 6 areused in combination. The quantization matrix table is set to have alow-frequency area that includes equal numbers of horizontal frequencycomponents and vertical frequency components, the area being larger thanthe corresponding area of the quantization matrix table in FIG. 8.

This provides a filter characteristic that removes neither horizontalnor vertical edges as noise.

FIG. 17 illustrates a quantization matrix table that is provided to thefilter circuit 2 when the two types of edge determination techniquesdescribed with reference to FIGS. 4 and 6 are used in combination. Thequantization matrix table is set to have a greater number of significantvertical frequency components than in the quantization matrix table inFIG. 11.

This provides a filter characteristic that prevents not only horizontaledges but also edges in general from being removed as noise.

FIG. 18 illustrates a quantization matrix table that is provided to thefilter circuit 2 when the two types of edge determination techniquesdescribed with reference to FIGS. 5 and 6 are used in combination. Thisquantization matrix table is set to include a greater number ofsignificant horizontal frequency components than in the quantizationmatrix table in FIG. 12.

This provides a filter characteristic that prevents not only verticaledges but also edges in general from being removed as noise.

Advantageous Effects

As described above, in the first preferred embodiment according to thepresent invention, the frequency analyzer 13 and the isolated frequencyanalyzer 14 detect edge components, and if edge components are detected,a weak filter is used in order not to remove those edges. It is thuspossible to prevent edge components from being removed through filteringand to suppress undesirable removal of significant components in theimage along with noise.

In addition, noise can be removed reliably because a strong filter isused when no edge components are detected.

Use of a noise filter that performs filtering on frequency componentsthat are obtained by decomposing image data suppresses undesirableremoval of significant components in the image along with noise in theprocess of noise removal.

Note that if the noise determination apparatus and the filter circuitdescribed above are mounted along with a codec on an image processingapparatus or the like, there is an effect of being able to determine thetypes of edge components and the presence or absence of noise with asimpler circuit configuration because the DCT unit and the quantizationunit of the codec can be shared.

Second Preferred Embodiment

FIG. 19 is a block diagram illustrating a configuration of anoise-removing apparatus 200 that includes a noise determinationapparatus according to a second preferred embodiment of the presentinvention.

As illustrated in FIG. 19, the noise-removing apparatus 200 includes anoise determination apparatus 1A that receives input of image data anddetermines whether or not the image data includes noise, a filtercircuit 2A that removes noise included in the image data, and aselection circuit 3A that selects and outputs either filtered image dataoutput from the filter circuit 2A or non-filtered image data, on thebasis of a result of noise determination performed by the noisedetermination apparatus 1A.

FIG. 20 is a block diagram illustrating a configuration of the noisedetermination apparatus 1A. As illustrated in FIG. 20, the noisedetermination apparatus 1A receives input of Bayer data serving as imagedata and temporarily stores the Bayer data in a line memory group 11A.Note that the line memory group 11A has the same functionality as theline memory group 11 in FIG. 2.

The Bayer data stored in the line memory group 11A is transmitted to aDCT unit 12A in units of 4×4 or 8×8 pixels in first-in, first-out order.The data is transmitted in succession until four or eight lines of theBayer data have been transmitted. Note that the DCT unit 12A has thesame functionality as the DCT unit 12 in FIG. 2.

The transform coefficients output from the DCT unit 12A are analyzed bya frequency analyzer 13A, and a result of the analysis indicatingwhether or not the image data in a unit of 4×4 or 8×8 pixels includesnoise is stored in an analysis result storage unit 14A that isconfigured by a memory, for example.

After being stored in the analysis result storage unit 14A, the resultof the analysis performed on one frame's worth of image data istransmitted to a single-frame noise determination unit 15A.

The single-frame noise determination unit 15A determines whether or notthe one frame's worth of image data is image data that includes noise,i.e., a frame that includes noise, on the basis of the number of Bayerdata blocks that have been determined to include noise according to theresult of the analysis performed on the one frame's worth of image data.

A result of the determination performed by the single-frame noisedetermination unit 15A is transmitted to a multiple-frame noisedetermination unit 16A. The multiple-frame noise determination unit 16Aperforms final noise determination on the basis of the number of framesthat include noise among a predetermined number of a plurality offrames, transmits a control signal based on a result of thedetermination to the selection circuit 3A, and controls the operation ofthe selection circuit 3A when selecting image data.

Note that the noise determination apparatus 1A is configured of a CPU, aRAM, a ROM, and a wired logic circuit, for example, and thefunctionality of the noise determination apparatus 1A is implemented bythe CPU executing a program that is stored in and read out of the ROM.

Noise Determination Processing

Next is a description of processing for determining noise, performed bythe noise determination apparatus 1A, with reference to the flowchart inFIG. 21. The following describes processing from the frequency analyzer13A onward in FIG. 20, and a description of DCT processing, which is aknown technique, will be omitted.

First, in step S41, the frequency analyzer 13A normalizes alternatingcurrent (AC) components in a matrix (corresponding to a Bayer data blockof 8×8 pixels) of transform coefficients (DCT coefficients) output fromthe DCT unit 12A by dividing the AC components by a direct current (DC)component in the matrix.

Here, FIG. 22 illustrates a matrix of DCT coefficients that correspondsto a Bayer data block of 8×8 pixels. In FIG. 22, the first element (atthe upper left corner) of the matrix is the DC component, elements alongthe horizontal direction of the matrix are vertical frequency components(V), and elements along the vertical direction of the matrix arehorizontal frequency components (H). AC components that are closer tothe DC component have lower frequencies, and AC components that arecloser to a corner that is diagonally opposite to the DC component inthe matrix have higher frequencies.

In FIG. 22, a sampling area of low-frequency components that are ACcomponents located close to the DC component is shown as a sampling areaLSR (low-frequency sampling area), and a sampling area of high-frequencycomponents that are AC components located close to the corner diagonallyopposite to the DC component is shown as a sampling area HSR(high-frequency sampling area).

Next, in step S42, the frequency analyzer 13A calculates a ratio (Σ highfrequencies/Σ low frequencies) of the total number of DCT coefficientsnormalized in the sampling area HSR (Σ high frequencies) to the totalnumber of DCT coefficients normalized in the sampling area LSR (Σ lowfrequencies) in FIG. 22, and compares the calculated ratio with apredetermined threshold value D. If the ratio exceeds the thresholdvalue D, the procedure proceeds to step S43. If the ratio is less thanor equal to the threshold value D, it is determined that the Bayer datablock to be processed does not include noise, and a result of thedetermination is stored in the analysis result storage unit 14A (stepS51). The procedure then proceeds to step S45.

Note that in FIG. 22, the number of frequency components in the samplingarea LSR and the number of frequency components in the sampling area HSRare both set to be approximately 30 percent of the total number offrequency components. This is because too large sampling areas can causedifficulty in obtaining a significant result. However, basically thenumber of frequency components in the sampling areas can be arbitrarilyset.

In step S43, the total number of DCT coefficients normalized in thesampling area HSR (Σ high frequencies) is compared with a predeterminedthreshold value E. This measure is taken in order to cope with thepossibility that a small number of high frequency components in amonotonous image, such as an image that includes a greater number of DCcomponents than AC components, will be misrecognized as noise in thedetermination process performed in step S42. If a result of thecomparison in step S43 shows that the number of high-frequencycomponents exceeds the predetermined number, it is determined that thedetermination in step S42 is correct.

If it is determined in step S43 that the total number of DCTcoefficients normalized in the sampling area HSR exceeds the thresholdvalue E, it is determined that the Bayer data block to be processedincludes noise, and a result of the determination is stored in theanalysis result storage unit 14A (step S44).

Next, in step S45, the frequency analyzer 13A determines whether or notthe noise determination processing has been performed on all Bayer datablocks that constitute the single frame. If the noise determinationprocessing has been performed on all of the Bayer data blocks, theprocedure proceeds to step S46, and if there is another Bayer data blockthat has not yet undergone the determination processing, the processingfrom step S41 onward is repeated.

In step S46, the single-frame noise determination unit 15A determines,on the basis of the determination result stored in the analysis resultstorage unit 14A, whether or not the number of Bayer data blocks thathave been determined to include noise among the one frame's worth ofBayer data is greater than a predetermined threshold value F. If it isdetermined that the number of Bayer data blocks that include noise isgreater than the predetermined threshold value F, it is determined thatthe corresponding frame includes noise, and a result of thedetermination is stored in the analysis result storage unit 14A (stepS47). On the other hand, if it is determined in step S46 that the numberof Bayer data blocks that include noise is less than or equal to thepredetermined threshold value F, it is determined that the correspondingframe does not include noise, and a result of the determination isstored in the analysis result storage unit 14A (step S52).

In this case, the threshold value F can be set to an arbitrary number.For example, a frame in which 30% of the total number of Bayer datablocks includes noise may be determined as a frame that includes noise.

Then, the multiple-frame noise determination unit 16A confirms, forexample, the number of frames that have been determined to include noisein step S47 and a continuous number of frames that have been determinedto include noise, and determines whether or not the plurality of framessatisfy a predetermined specified determination condition (step S48). Ifthe frames satisfy the determination condition, the multiple-frame noisedetermination unit 16A determines that a moving image includes noise(step S49), and transmits a predetermined control signal to theselection circuit 3A so that data from which noise has been removed isoutput (step S50). Thereafter, the processing from step S41 onward isrepeated until there is no more input of image data, such as in the casewhere the noise-removing apparatus 200 is turned off.

On the other hand, if it is determined in step S48 that the frames donot satisfy the specified determination condition, the multiple-framenoise determination unit 16A determines that the moving image does notinclude noise (step S53), and transmits a predetermined control signalto the selection circuit 3A (step S50). Note that the control signal isoutput in either case where noise is determined to be present or wherenoise is determined not to be present, and set to, for example, “0” whennoise is present and to “1” when noise is not present. The selectioncircuit 3A performs a selection operation corresponding to the controlsignal.

Now, the determination process performed in step S48 will be described.A moving image typically consists of 60 frames of images per second.Thus, the determination in step S48 is started after the processing fromsteps S41 to S47 and step S52 has been performed on 60 frames of imagedata. The determination process uses a determination condition that ifmore than half of 60 frames of a moving image include noise, the movingimage is determined to include noise, and in this case, a control signalfor controlling the selection operation of the selection circuit 3A isoutput so that as the 61th and subsequent frames of image data, filteredimage data from which noise has been removed by the filter circuit 2A isoutput through the selection circuit 3A.

In other words, non-filtered image data has been output through theselection circuit 3A until the 60 frames of image data that arenecessary for the final determination are collected.

Although the 61th and subsequent frames of image data are filtered imagedata, if the number of frames that do not include noise increases andthe above-described determination condition becomes not satisfied in theimaging situation after repetition of the processing from steps S41 toS53, non-filtered image data will be output through the selectioncircuit 3A.

The reason for the above-described processing is to avoid occurrence offlicker in captured images due to frequent switching on and off offiltering, although performing filtering and not performing filteringcause a great difference in resultant captured images.

Note that the above-described determination condition is not limited tothe above number of frames that have been determined to include noise,and may be such that if a predetermined number of frames that have beendetermined to include noise is continuous, e.g., more than 10 frames outof 60 frames are continuous, the moving image is determined to include anoise, and a control signal for controlling the selection operation ofthe selection circuit 3A is output so that for the 61th and subsequentframes of image data, filtered image data from which noise has beenremoved by the filter circuit 2A is output through the selection circuit3A.

Here, the accuracy of determination can be improved by performing theabove-described processing from steps S41 to S43 on each of four colorsof input Bayer data, namely, green of blue side (Gb), green of red side(Gr), red (R), and blue (B). Alternatively, the above-describedprocessing from steps S41 to S43 may be performed on only Gb or Gr databecause the green components include luminance information.

Variation of Noise Determination Apparatus

The following describes a variation of the noise determination apparatus1A. FIG. 23 is a block diagram of a noise determination apparatus 1B asa variation of the noise determination apparatus 1A.

As illustrated in FIG. 23, the noise determination apparatus 1B receivesinput of Bayer data serving as image data and temporarily stores theBayer data in the line memory group 11A.

The Bayer data stored in the line memory group 11A is transmitted to theDCT unit 12A in units of 4×4 or 8×8 pixels in first-in, first-out order.The data is transmitted in succession until four or eight lines of Bayerdata have been transmitted.

The transform coefficients output from the DCT unit 12A are analyzed bythe frequency analyzer 13A, and a result of the analysis indicatingwhether or not the image data in a unit of 4×4 or 8×8 pixels includesnoise is stored in the analysis result storage unit 14A.

After being stored in the analysis result storage unit 14A, the resultof the analysis performed on the one frame's worth of image data istransmitted to the multiple-frame noise determination unit 16A. Themultiple-frame noise determination unit 16A performs final noisedetermination on the basis of the number of frames that include noiseamong a predetermined number of a plurality of frames, transmits acontrol signal based on a result of the determination to the selectioncircuit 3A, and controls the operation of the selection circuit 3A whenselecting image data.

Next is a description of processing for determining noise, performed bythe noise determination apparatus 1B, with reference to the flowchart inFIG. 24.

First, in step S61, the frequency analyzer 13A normalizes alternatingcurrent (AC) components in a matrix (corresponding to a Bayer data blockof 8×8 pixels) of transform coefficients (DCT coefficients) output fromthe DCT unit 12A by dividing the AC components by a direct current (DC)component in the matrix.

Next, in step S62, the frequency analyzer 13A adds DCT coefficients ofcorresponding frequency components in the matrices of normalized DCTcoefficients for all of the Bayer data blocks that constitute the singleframe. This produces a single matrix (data-integrated matrix) in whichthe DCT coefficients in all of the matrices are integrated.

Next, the frequency analyzer 13A calculates a ratio (Σ highfrequencies/Σ low frequencies) of the total number of DCT coefficientsnormalized in the high-frequency sampling area (Σ high frequencies) tothe total number of DCT coefficients normalized in the low-frequencysampling area (Σ low frequencies) in the data-integrated matrix, andcompares the calculated ratio with a predetermined threshold value D. Ifthe ratio exceeds the threshold value D, the procedure proceeds to stepS64. If the ratio is less than or equal to the threshold value D, it isdetermined that the frame to be processed does not include noise, and aresult of the determination is stored in the analysis result storageunit 14A (step S69). The procedure then proceeds to step S66.

In step S64, the frequency analyzer 13A compares the total number of DCTcoefficients normalized in the high-frequency sampling area (Σ highfrequencies) with a predetermined threshold value E. This measure istaken in order to cope with the possibility that a small number ofhigh-frequency components in a monotonous image, such as an image thatincludes a greater number of DC components than AC components, will bemisrecognized as noise in the determination process performed in stepS63. If a result of the comparison in step S64 shows that the number ofhigh-frequency components exceeds the predetermined number, it isdetermined that the determination in step S63 is correct.

If it is determined in step S64 that the total number of DCTcoefficients normalized in the high-frequency sampling area exceeds thethreshold value E, it is determined that the frame to be processedincludes noise, and a result of the determination is stored in theanalysis result storage unit 14A (step S65).

Then, the multiple-frame noise determination unit 16A confirms, forexample, the number of frames that have been determined to include noiseand stored in step S65 and a continuous number of frames that have beendetermined to include noise, and determines whether or not the pluralityof frames satisfy a predetermined specified determination condition(step S66). If the frames satisfy the determination condition, themultiple-frame noise determination unit 16A determines that a movingimage includes noise (step S67), and transmits a predetermined controlsignal to the selection circuit 3A so that data from which noise hasbeen removed is output (step S68). Thereafter, the processing from stepS61 onward is repeated until there is no more input of image, such as inthe case where the noise-removing apparatus 200 is turned off.

On the other hand, if it is determined in step S66 that the frames donot satisfy the predetermined determination condition, themultiple-frame noise determination unit 16A determines that the movingimage does not include noise (step S70), and transmits a predeterminedcontrol signal to the selection circuit 3A (step S28). Note that thecontrol signal is output in either case where noise is determined to bepresent or where noise determined not to be present, and set to, forexample, “0” when noise is present and to “1” when noise is not present.The selection circuit 3A performs a selection operation corresponding tothe control signal.

Note that the determination operation performed in step S66 is the sameas that performed in step S48, which is previously described withreference to FIG. 21.

Here, the accuracy of determination can be improved by performing theabove-described processing from steps S61 to S64 on each of four colorsof input Bayer data, namely, green of blue side (Gb), green of red side(Gr), red (R), and blue (B). Alternatively, the above-describedprocessing from steps S61 to S64 may be performed on only Gb or Gr databecause the green components include luminance information.

Note that the above-described noise determination apparatuses 1A and 1Bmay each be used independently, or they may be used in parallel witheach other for synthetic noise determination based on the results ofnoise determination processing performed by the two apparatuses.

In other words, the noise determination processing performed by thenoise determination apparatus 1A, which determines noise in units ofBayer data blocks, has the feature of being able to accurately determinenoise even in a complicated image such as an image that includes alopsided distribution of areas that are difficult to determine noise.

On the other hand, the noise determination processing performed by thenoise determination apparatus 1B, which integrates all areas of thesingle frame so as to determine noise in the integrated single Bayerdata block, has the feature of being suited to use for a simple image,such as an image that has a uniform distribution of noise, because onlya small number of processing steps is necessary and noise can bedetermined relatively easily.

Accordingly, using both of the noise determination apparatuses 1A and 1Bmakes it possible to combine the above-described two features and tocomplement each other's drawbacks, thus enabling accurate noisedetermination.

Setting of Sampling Areas

While the numbers of frequency components in the sampling areas can beset arbitrarily as described previously, they may be set based on theaverage value of Gb and Gr data in the entire frame.

In other words, since the green components include luminanceinformation, setting the sampling areas according to the luminance(brightness) of the image will achieve the following effects.

For example, when the luminance of the image is high, the number ofsamples can be reduced because the S/N ratio is high. Thus, the samplingareas LSR and HSR are reduced as illustrated in FIG. 25. Reducing thesampling areas will reduce the amount of data to be processed andincrease the processing speed.

On the other hand, when the luminance of the image is low, the samplingareas LSR and HSR are both increased as illustrated in FIG. 26 becausethe S/N ratio is low. Increasing the number of samples will suppress areduction in the accuracy of determination.

Note that while FIGS. 22, 25, and 26 illustrate examples in which thesampling areas LSR and HSR each include equal numbers of high- andlow-frequency components, each of the sampling areas may includedifferent numbers of high- and low-frequency components.

Advantageous Effects

As described above, the second preferred embodiment according to thepresent invention allows the presence or absence of noise to berelatively simply and easily determined because the frequency analyzer13A analyzes the presence or absence of noise by calculating the ratioof Σ high frequencies to Σ low frequencies. Since Bayer data is used asimage data to determine noise, the accuracy of noise detection can beimproved as compared to the case of using data that is generated byfetching Bayer data and performing color conversion on the Bayer datathrough interpolation processing.

Note that if the noise determination apparatuses described above aremounted along with a codec on an image processing apparatus or the like,there is an effect of being able to determine the presence or absence ofnoise with a simpler circuit configuration because the DCT unit and thequantization unit of the codec can be shared.

Other Exemplary Applications

While the above-described first and second preferred embodiments of thepresent invention explain configurations for determining noise usingBayer data as image data, YCbCr data or YUV data may be used todetermine noise.

While the above description takes the example of using a DCT forfrequency decomposition, other transform techniques such as a FastFourier Transform (FFT) or a wavelet transform may be used in the noisedetermination processing.

Note that preferred embodiments of the invention can appropriately bemodified or partly omitted in various ways without departing from thescope of the invention.

While the invention has been shown and described in detail, theforegoing description is in all aspects illustrative and notrestrictive. It is therefore understood that numerous modifications andvariations can be devised without departing from the scope of theinvention.

What is claimed is:
 1. A noise determination apparatus for determiningnoise in image data that is input in units of frames and a significantedge in an image, said noise determination apparatus comprisingcircuitry configured to: decompose said image data into frequencycomponents: sample a predetermined number of data pieces forlow-frequency components that have relatively low frequencies and apredetermined number of data pieces for high-frequency components thathave relatively high frequencies from the frequency components; anddetermine the noise or the significant edge in the image, on the basisof a ratio of high-frequency data to low-frequency data.
 2. The noisedetermination apparatus according to claim 1, wherein said circuitry isconfigured to divide one frame's worth of said image data into matricesof a predetermined number of pixels, the matrices serving as datablocks, and decompose each data block into frequency components; samplea predetermined number of low-frequency components and a predeterminednumber of high-frequency components from the frequency components ofsaid data block to use the low-frequency components as saidlow-frequency data and use the high-frequency components as saidhigh-frequency data, and analyze whether or not said data block includesan edge image, on the basis of a ratio of said high-frequency data tosaid low-frequency data; and select a filter characteristic of a filterthat removes noise, on the basis of a result of the analysis performed.3. The noise determination apparatus according to claim 2, wherein thecircuitry is configured to set a first sampling area in which saidlow-frequency data includes a greater number of horizontal frequencycomponents than vertical frequency components, in a case where a ratioof said high-frequency data to said low-frequency data in said firstsampling area is less than or equal to a predetermined threshold value,analyze that said data block includes a horizontal edge image, andselect a filter characteristic that does not remove said horizontal edgeimage.
 4. The noise determination apparatus according to claim 2,wherein the circuitry is configured to set a second sampling area inwhich said low-frequency data includes a greater number of verticalfrequency components than horizontal frequency components, in a casewhere a ratio of said high-frequency data to said low-frequency data insaid second sampling area is less than or equal to a predeterminedthreshold value, analyze that said data block includes a vertical edgeimage, and select a filter characteristic that does not remove saidvertical edge image.
 5. The noise determination apparatus according toclaim 2, wherein the circuitry is configured to set a third samplingarea in which said low-frequency data includes equal numbers of verticalfrequency components and horizontal frequency components, in a casewhere a ratio of said high-frequency data to said low-frequency data insaid third sampling area is less than or equal to a predeterminedthreshold value, analyze that said data block includes an edge image,and select a filter characteristic that does not remove said edge image.6. The noise determination apparatus according to claim 2, wherein thecircuitry is configured to decompose a data block into frequencycomponents through a discrete cosine transform, and normalizealternating current components among transform coefficients obtainedthrough the discrete cosine transform by dividing said alternatingcurrent components by a direct current component among said transformcoefficients, and use the normalized alternating current components assaid low-frequency data and said high-frequency data.
 7. The noisedetermination apparatus according to claim 2, wherein said circuitry isconfigured to analyze whether or not a frequency component is isolatedfrom other frequency components to determine presence or absence of anisolated frequency, analyze whether or not said data block includes anedge image that is unable to be analyzed, on the basis of the presenceor absence of said isolated frequency, and in a case where said datablock includes an edge image that is unable to be analyzed, select afilter characteristic that does not remove said edge image.
 8. The noisedetermination apparatus according to claim 7, wherein the circuitry isconfigured to decompose a data block into frequency components through adiscrete cosine transform, and normalize alternating current componentsamong transform coefficients that are obtained through the discretecosine transform by dividing said alternating current components by adirect current component among said transform coefficients, calculate anaverage value of all normalized alternating current components,calculate, for each of frequency components of said normalizedalternating current components, the number of surrounding frequencycomponents that have values greater than said average value, calculatethe number of frequency components for which the calculated number ofsurrounding frequency components is less than a predetermined firstthreshold value, determine whether or not the calculated number offrequency components is less than a predetermined second thresholdvalue, and if the calculated number of frequency components is less thansaid second threshold value, analyze that said data block includes anedge image that is unable to be analyzed.
 9. The noise determinationapparatus according to claim 1, wherein Bayer data serves as said imagedata.
 10. A noise determination apparatus for determining presence orabsence of noise in image data that is input in units of frames, saidnoise determination apparatus comprising circuitry configured to:decompose said image data into frequency components; sample apredetermined number of data pieces for low-frequency components thathave relatively low frequencies and a predetermined number of datapieces for high-frequency components that have relatively highfrequencies from the frequency components; and determine the presence orabsence of noise on the basis of a ratio of high-frequency data tolow-frequency data.
 11. The noise determination apparatus according toclaim 10, wherein said circuitry is configured to divide one frame'sworth of said image data into matrices of a predetermined number ofpixels, the matrices serving as data blocks, and decompose each datablock into frequency components; and sample a predetermined number oflow-frequency components and a predetermined number of high-frequencycomponents from the frequency components of said data block to use thelow-frequency components as said low-frequency data and use thehigh-frequency components as said high-frequency data, and analyze thepresence or absence of noise on the basis of whether or not a ratio ofsaid high-frequency data to said low-frequency data is greater than apredetermined first threshold value.
 12. The noise determinationapparatus according to claim 11, wherein said circuitry is configured todetermine that said one frame's worth of said image data includes noise,in a case where the number of said data blocks that are determined toinclude noise is greater than a predetermined threshold value.
 13. Thenoise determination apparatus according to claim 12, wherein saidcircuitry is configured to determine that a moving image includes noise,in a case where a plurality of frames that are determined to includenoise satisfy a predetermined determination condition.
 14. The noisedetermination apparatus according to claim 10, wherein said circuitry isconfigured to divide one frame's worth of said image data into matricesof a predetermined number of pixels, the matrices serving as datablocks, and decompose each data block into frequency components; andsample a predetermined number of low-frequency components and apredetermined number of high-frequency components from the frequencycomponents of said data block to use the low-frequency components assaid low-frequency data and use the high-frequency components as saidhigh-frequency data, generate a data-integrated matrix by adding datapieces of corresponding frequency components in all of said data blocksthat constitute said one frame's worth of said image data, and analyzethe presence or absence of noise on the basis of whether or not a ratioof said high-frequency data to said low-frequency data in saiddata-integrated matrix is greater than a predetermined first thresholdvalue.
 15. The noise determination apparatus according to claim 14,wherein said circuitry is configured to determine that a moving imageincludes noise, in a case where a plurality of frames that aredetermined to include noise satisfies a predetermined determinationcondition.
 16. The noise determination apparatus according to claim 13,wherein said determination condition is defined by a ratio of framesthat include noise to a plurality of frames that constitute a movingimage.
 17. The noise determination apparatus according to claim 15,wherein said determination condition is defined by a ratio of framesthat include noise to a plurality of frames that constitute a movingimage.
 18. The noise determination apparatus according to claim 13,wherein said determination condition is defined by a continuous numberof frames that include noise among a plurality of frames that constitutea moving image.
 19. The noise determination apparatus according to claim15, wherein said determination condition is defined by a continuousnumber of frames that include noise among a plurality of frames thatconstitute a moving image.
 20. The noise determination apparatusaccording to claim 11, wherein the circuitry is configured to decomposea data block into frequency components through a discrete cosinetransform, and normalize alternating current components among transformcoefficients obtained through the discrete cosine transform by dividingsaid alternating current components by a direct current component amongsaid transform coefficients, and use the normalized alternating currentcomponents as said low-frequency data and said high-frequency data. 21.The noise determination apparatus according to claim 20, wherein saidcircuitry is configured to analyze the presence or absence of noise byfurther determining whether or not said high-frequency data that hasbeen normalized is greater than a predetermined second threshold value.22. The noise determination apparatus according to claim 14, wherein thecircuitry is configured to decompose a data block into frequencycomponents through a discrete cosine transform, and normalizealternating current components among transform coefficients that areobtained through the discrete cosine transform by dividing saidalternating current components by a direct current component among saidtransform coefficients, and use the normalized alternating currentcomponents as said low-frequency data and said high-frequency data. 23.The noise determination apparatus according to claim 22, wherein thecircuitry is configured to analyze the presence or absence of noise byfurther determining whether or not said high-frequency data that hasbeen normalized is greater than a predetermined second threshold value.24. The noise determination apparatus according to claim 10, whereinBayer data serves as said image data.
 25. A noise determination methodperformed by a noise determination apparatus for determining noise inimage data that is input in units of frames and a significant edge in animage, the method comprising the steps of: (a) decomposing said imagedata into frequency components; and (b) sampling a predetermined numberof data pieces for low-frequency components that have relatively lowfrequencies and a predetermined number of data pieces for high-frequencycomponents that have relatively high frequencies from the frequencycomponents obtained in said step (a), and determining the noise or thesignificant edge in the image, on the basis of a ratio of high-frequencydata to low-frequency data.
 26. A noise determination method fordetermining presence or absence of noise in image data that is input inunits of frames, the method comprising inputting the image data intocircuitry; decomposing, in the circuitry, said image data into frequencycomponents; and sampling, in the circuitry, a predetermined number ofdata pieces for low-frequency components that have relatively lowfrequencies and a predetermined number of data pieces for high-frequencycomponents that have relatively high frequencies from the frequencycomponents obtained in said decomposing, and determining, in thecircuitry, the presence or absence of noise on the basis of a ratio ofhigh-frequency data to low-frequency data.